-
Notifications
You must be signed in to change notification settings - Fork 329
feat(transaction): Implement TransactionAction for updata_loc, update_props, and upgrade_format #1433
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
…rmat_version, add as_any to tx_action trait
pub(crate) trait TransactionAction: Sync + Send { | ||
/// Returns the action as [`Any`] so it can be downcast to concrete types later | ||
fn as_any(self: Arc<Self>) -> Arc<dyn Any>; | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I added this mainly for testing/inspecting the content of applied TransactionAction
s
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This could be a provided method.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I played with this a little bit but it seems like this has to be done within concrete implementations. With code like below:
fn as_any(self: Arc<Self>) -> Arc<dyn Any> {
self
}
And this will fail to the error:
|
43 | self
| ^^^^ doesn't have a size known at compile-time
It seems that even if self is an Arc<T>
, the contained T
still has to be Sized
to be cast to Any
in a provided method. Adding where Self: Sized
bound to the method directly will make it uncallable on Arc<dyn TxAction>
so it won't work as well
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How about using as_any? This way we don't need to implement this for each action:
pub(crate) trait TransactionAction: AsAny {
...
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks Renjie, I gave it a try and it's working fine except some syntax sugar to use it on Arc:
let action = (*tx.actions[0]) // this
.downcast_ref::<UpdateLocationAction>()
.unwrap();
This is still much cleaner than using Any
directly!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @CTTY for this pr, generally looks good!
Co-authored-by: Renjie Liu <liurenjie2008@gmail.com>
Co-authored-by: Renjie Liu <liurenjie2008@gmail.com>
pub(crate) trait TransactionAction: Sync + Send { | ||
/// Returns the action as [`Any`] so it can be downcast to concrete types later | ||
fn as_any(self: Arc<Self>) -> Arc<dyn Any>; | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This could be a provided method.
Co-authored-by: Renjie Liu <liurenjie2008@gmail.com>
pub(crate) trait TransactionAction: Sync + Send { | ||
/// Returns the action as [`Any`] so it can be downcast to concrete types later | ||
fn as_any(self: Arc<Self>) -> Arc<dyn Any>; | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How about using as_any? This way we don't need to implement this for each action:
pub(crate) trait TransactionAction: AsAny {
...
}
Co-authored-by: Renjie Liu <liurenjie2008@gmail.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @CTTY for this pr, LGTM!
} | ||
|
||
/// Update table's property. | ||
pub fn set_properties(mut self, props: HashMap<String, String>) -> Result<Self> { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ooops, it's an interface change, one minor suggestion, better to call it out in PR description and later release notes. :)
It actually breaks moonlink compilation :(
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, we are making TransactionAction
s retryable and had to change the existing APIs. There are more API changes coming for other actions like ReplaceSortOrderAction
(#1441) and FastAppendAction
(WIP)
We definitely need to call this out in the next release notes. cc: @liurenjie1024
I'm not very familiar with moonlink and here goes the dumb question: Would it be better for moonlink to depend on a released/stable iceberg-rs version instead of tracking an unreleased git rev? You can set up a non-blocking github CI to track iceberg-rs/main
if you want to catch compatibility issues early
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We definitely need to call this out in the next release notes.
Thanks!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not very familiar with moonlink and here goes the dumb question: Would it be better for moonlink to depend on a released/stable iceberg-rs version instead of tracking an unreleased git rev?
That's the ideal case, but I think the interval between iceberg-rust releases are too long.
You can set up a non-blocking github CI to track iceberg-rs/main if you want to catch compatibility issues early
It's not too much of an overhead as of now, because I sync iceberg-rust upstream on weekly basic.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The release interval certainly is an issue 😄
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @dentiny for this comment. I thought about keeping the original api as deprecated, but the concern was to the overhead of maintaining it. But I do agree that we should shoutout this breaking change when release.
Which issue does this PR close?
Related Issues:
Transaction::commit
method #1387TableMetadata
to new location. #1388What changes are included in this PR?
TransactionAction
UpdateLocationAction
UpdatePropertiesAction
UpgradeFormatVersionAction
as_any
toTransactionAction
traitdo_commit
inTransaction
to commit applied actionsWe will also need to implement
TransactionAction
forFastAppendAction
andReplaceSortOrderAction
, and I'm planning to do that in a separate PR.Are these changes tested?
Added unit tests